Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Kobzol · 2025-08-20T15:15:22Z

This PR cleans up a bunch of test steps and adds metadata to them. I didn't yet touch the most complicated step (CompileTest), I'm leaving that for another PR.

Testing anything on stage 0 is only possible for compiletest and with build.allow-compiletest-stage0. Testing anything else on stage 0 will either produce a nice error or crash with a stage being subtracted below zero.

r? @jieyouxu

try-job: dist-x86_64-linux
try-job: aarch64-gnu
try-job: arm-android
try-job: x86_64-gnu-llvm-20*
try-job: x86_64-msvc-*
try-job: aarch64-apple
try-job: test-various

rustbot · 2025-08-20T15:16:02Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-08-20T15:19:54Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

Kobzol · 2025-08-20T17:37:55Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T17:51:18Z

💔 Test for 06dca67 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

Kobzol · 2025-08-20T17:53:44Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T18:01:48Z

💔 Test for 71ba459 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

rustbot · 2025-08-27T09:41:02Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kobzol · 2025-08-27T09:41:19Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple try-job: test-various

Kobzol · 2025-08-27T12:37:44Z

Ok, should be ready for a review now.

@rustbot ready

rust-bors · 2025-08-27T12:46:44Z

☀️ Try build successful (CI)
Build commit: d47e319 (d47e319edabed56167a4e98e0b0e18f8f415a7ff, parent: 4f808ba6bf9f1c8dde30d009e73386d984491587)

jieyouxu

Most changes look fine, I do have a question about the lint-docs / compiler doc tests

View changes since this review

jieyouxu · 2025-08-29T09:34:31Z

compiler/rustc_lint_defs/src/builtin.rs

+    #[cfg_attr(not(bootstrap), doc = "```")]
+    #[cfg_attr(bootstrap, doc = "```")]
+    // ^ Needed to avoid tidy warning about odd number of backticks


Question: is this because we run doctests on rustc_lint_defs with a build compiler, which could be a stage 0 rustc which happens to be a beta rustc?

Would this be broken under local-rebuild if the build compiler happens to be the in-tree compiler that e.g. would know about feature(sanitize)? Or is that not a supported configuration for this particular case?

Actually I think that we can remove this now, because in the latest commit I skipped the lint-docs tests in stage 1, and running them with explicit --stage 1 runs into other failures anyway. I'll try to remove it and see what CI says.

jieyouxu · 2025-08-29T10:30:35Z

src/bootstrap/src/core/build_steps/test.rs

@@ -3023,7 +3136,7 @@ impl Step for CrateRustdocJsonTypes {
        );

        // FIXME: this looks very wrong, libtest doesn't accept `-C` arguments and the quotes are fishy.
-        let libtest_args = if self.host.contains("musl") {
+        let libtest_args = if target.contains("musl") {


Remark (not for this PR): wait, I think I wrote

// FIXME: this looks very wrong, libtest doesn't accept `-C` arguments and the quotes are fishy.

jieyouxu · 2025-08-29T10:35:53Z

src/bootstrap/src/core/build_steps/test.rs

+        // We need to run the cranelift tests with the compiler against cranelift links to, not with
+        // the build compiler.


Nit: I think this meant to say

We need to run the craftlift tests with the compiler that cranelift links against, not with the build compiler

?

jieyouxu · 2025-08-29T10:39:18Z

src/bootstrap/src/core/builder/tests.rs

+        [test] Crashes <host>
+        [build] rustc 0 <host> -> CoverageDump 1 <host>
+        [build] rustc 1 <host> -> std 1 <host>
+        [test] CodegenLlvm <host>


Remark: ah, compiletest ones are intentionally out of scope.

src/ci/docker/host-x86_64/tidy/Dockerfile

jieyouxu · 2025-08-29T10:45:18Z

src/tools/lint-docs/src/lib.rs

@@ -234,6 +229,20 @@ impl<'a> LintExtractor<'a> {
                            // Ignore allow of lints (useful for
                            // invalid_rust_codeblocks).
                            continue;
+                        } else if let Some(text) =
+                            line.strip_prefix("#[cfg_attr(not(bootstrap), doc = \"")


Question: wait, is this now needed because the compiler that is used to build and run lint tools might be a different build compiler, versus the compiler under test?

jieyouxu · 2025-08-29T10:47:27Z

@rustbot author

rustbot assigned jieyouxu Aug 20, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 20, 2025

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

rustbot added A-CI Area: Our Github Actions CI T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 20, 2025

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-test branch from 00e0ffb to c501499 Compare August 20, 2025 15:34

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-test branch from 8e5d2e3 to 9887c77 Compare August 20, 2025 16:52

This comment has been minimized.

Sign in to view

Kobzol added 17 commits August 27, 2025 11:40

Remove compiler_for from test::CodegenCranelift

1f81491

Refactor test::LintDocs

2f1bbc6

Remove stage0 checking

df99001

Small refactoring around RemoteCopyLibs

92e9d25

Add metadata to a bunch of steps, rename variables and add comments

b977586

Move the test! macro closer to its usages

0a27573

Fix staging for compiler/std crate tests

d598209

Fix spacing when testing individual crates

85274eb

Do not run tests on CI in stage 0

4728ef4

Add change tracker entry

d1e94e1

Skip bootstrap tests on CI

6c93dfb

Fix doclink

a9b85e9

Fake nodejs in snapshot test

4da586f

Fix tests and doclink

c8ae70a

Add test stage 2 snapshot tests

ee109d3

Fix stage 1 compiler tests

a000297

Do not run lint-docs tests in stage 1 x test by default

9a75636

Kobzol force-pushed the bootstrap-test branch from 4274d40 to 9a75636 Compare August 27, 2025 09:40

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 27, 2025

jieyouxu reviewed Aug 29, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 29, 2025

		// We need to run the cranelift tests with the compiler against cranelift links to, not with
		// the build compiler.

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Are you sure you want to change the base?

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Conversation

Kobzol commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

This comment has been minimized.

rustbot commented Aug 27, 2025

Uh oh!

Kobzol commented Aug 27, 2025

Uh oh!

This comment has been minimized.

Kobzol commented Aug 27, 2025

Uh oh!

rust-bors bot commented Aug 27, 2025

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jieyouxu Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Aug 29, 2025

Uh oh!

Uh oh!

Kobzol commented Aug 20, 2025 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading